In recent years, systems utilizing image communication such as a visual telephone and a television conference system has been becoming of great interest. Typically, a communication line utilized in this system has a low transmission speed. Therefore, an image encoding/decoding technique is essential for the transmission of image data of massive data amount, and actually, various kinds of encoding/decoding methods have been devised.
Further, in these circumstances, it has been desired to provide an image processing apparatus which performs image encoding/decoding processing by a processor which can be controlled by programs, and which can flexibly correspond to various encoding/decoding methods by replacement of a control program.
Further, in video compression and decompression processings in the conventional image processing apparatus, complicated data transfer, such as obtaining image data of a part of a region, has been required. Such complicated data transfer is realized on the basis of a large amount of control information. Therefore, there is a problem that, after the control information is obtained after the data transfer which is presently being executed is completed, the next data transfer is executed, the larger the number of control information is, the longer the time interval between the completion of the data transfer and the start of the next data transfer is, thereby resulting in a deteriorated data transfer efficiency.
In order to solve this problem, in the conventional image processing apparatus, there is provided a data transfer control apparatus in which there are provided data transfer control information obtaining means and reservation data transfer information holding means, the control information required for the next data transfer is acquired while the data transfer is being executed to be stored in the reservation data transfer information holding means, and when the data transfer now being executed is completed, the next data transfer can be immediately started on the basis of the control data prepared in the reservation data transfer information holding means.
An example of an image processing apparatus X which is conventionally used as described above and performs encoding/decoding processing by a processor which can be controlled by programs will be described with reference to FIG. 8.
FIG. 8 is a structural block diagram illustrating the image processing apparatus X. The image processing apparatus X includes image input/output processing means 500, input control means 501 for performing resolution conversion of the input image and image noise removal processing using the input image and the previous frame image, an input image buffer 502 which temporarily holds the output image data of the input control means 501 before transferring the data from the input control means 501 to an external memory 511, a previous frame image buffer 503 which temporarily holds the previous frame image data stored in the external memory 511 before giving the same to the input control means 501, a sub-picture generation input buffer 504 which temporarily holds the display image data stored in the external memory 511 before giving the same to sub-picture generating means 505, sub-picture generating means 505 for performing resolution conversion of the display image data stored in the external memory 511 into the sub-picture, a sub-picture generation output buffer 506 which temporarily holds sub-picture image data which is generated by the sub-picture generating means 505, before transferring the data from the sub-picture generating means 505 to the external memory 511, a sub-picture buffer 507 which temporarily holds the sub-picture image data stored in the external memory 511, before giving the same to display control means 510, a main picture buffer 508 which temporarily holds the display image data which is stored in the external memory 511, before giving the same to the display control means 510, a graphics buffer 509 which temporarily holds graphics data stored in the external memory 511, before giving the same to the display control means 510, an external memory 511 which stores data, with performing region segmentation for different types of image data, such as encoding target image, a sub-picture, a display image, and graphics, DMA control means 512 for controlling a data transfer between the image input/output processing means 500 or the processor means 516 and the external memory 511, i.e., a Direct Memory Access (hereinafter, referred to as “DMA”), DMA settings holding means 513 for holding respective setting information for performing the DMA control, address generating means 514 for generating an address of the external memory 511 in accordance with the DMA setting information, DRAM control means 515 for controlling writing and reading of the external memory 511, the processor means 516 which can be controlled by programs, encoding/decoding processing means 517 for processing an image of the external memory 511 or code data by program control, and a DMA bus 518 which performs DMA between the input image buffer 502, the previous frame image buffer 503, the sub-picture generation input buffer 504, the sub-picture generation output buffer 506, the sub-picture buffer 507, the main picture buffer 508, and the graphics buffer 509, the processor means 516, and the external memory 511.
Hereinafter, the operation of the image processing apparatus X so constructed will be briefly described.
Initially, an input image is always input to the input control means 501 with synchronized with a video synchronizing signal at a constant rate.
After the input image is input to the input control means 501, the input control means 501 performs resolution conversion of the input image into the image size to be encoded. Thereafter, the input control means 501 temporarily stores the same in the input image buffer 502.
In addition, the input control means 501 performs removal processing of noises in the input image by using the previous frame image. In this case, the previous frame image stored in the external memory 511 is subjected to the DMA to the previous image buffer 503, and noise removal processing is performed while reading the previous frame image from the previous frame image buffer 503.
When the display image stored in the external memory 511 is displayed as the sub-picture, the sub-picture generating means 505 performs resolution conversion of the display image transferred from the external memory 511 through the sub-picture generation input buffer 504 into the sub-picture size, and temporarily stores the same in the sub-picture generation output buffer 506. Thereafter, the stored display image is transferred from the sub-picture generation output buffer 506 to the external memory 511.
The display control means 510 reads the sub-picture, the main picture, and the graphics data from the sub-picture buffer 507, the main picture buffer 508, and the graphics buffer 509, respectively, and, after display composition, synchronizes the same with the video synchronizing signal as the display image to output at a constant rate.
The processor means 516 transfers image data of an encoding target, which is stored in the external memory 511, to the inside of the processor, performs encoding processing, and transfers the code data to the external memory 511. Further, the processor means 516 transfers the code data stored in the external memory 511 to the inside of the processor, performs decoding processing, and transfers the same to the external memory 511 as the display image data.
The DMA among the external memory 511, the input image buffer 502, the previous frame image buffer 503, the sub-picture generation input buffer 504, the sub-picture generation output buffer 506, the sub-picture buffer 507, the main picture buffer 508, and the graphics buffer 509, and the processor means 516 is executed by the processor means 516 making DMA request to the DMA control means 512.
When the DMA request is made by the processor means 516, the DMA control means 512 gives the DMA settings information which is set in DMA setting holding means 513 to the address generation means 514.
The address generating means 514 generates an access address of the external memory 511 on the basis of the received DMA setting information, and gives the same to the DRAM control means 515.
The DRAM control means 515 controls reading or writing of either of the input image buffer 502, the previous frame image buffer 503, the sub-picture generation input buffer 504, the sub-picture generation output buffer 506, the sub-picture buffer 507, the main picture buffer 508, and the graphics buffer 509, or the processor means 516, and writing or reading of the external memory 511.
The external memory 511, the input image buffer 502, the previous frame image buffer 503, the sub-picture generation input buffer 504, the sub-picture generation output buffer 506, the sub-picture buffer 507, the main picture buffer 508, the graphics buffer 509, and the processor buffer 516 are connected by a single DMA bus, and the DMA with the external memory 511 is performed by the time division method.
However, in a structure of the conventional image processing apparatus X as described above, all the DMA scheduling with the external memory 511 is performed by the processor means 516. Therefore, there has been a problem that the publication timing of the DMA request sometimes becomes irregular in accordance with a processing load of the processor means 516.
On the other hand, image input/output of image input/output processing means 500 must be synchronized with the video synchronizing signal and be input and output at the constant rate. Therefore, in order to absorb the irregularity of the DMA, the memory capacity of each buffer is increased, and the measures such as the speed-up of the transfer speed due to the extension of the DMA bus width and the raise of an operating frequency are taken. However, here, there has been a problem such as the increase in the circuit scale and the complication of the circuit design.
Further, in a data transfer control apparatus used in the conventional image processing apparatus, after completing the data transfer which is being executed, the next data transfer is executed. Therefore, there has been a problem, for example, that a transfer execution start of urgently required data is delayed, and therefore, data cannot be obtained within the required time.
The present invention is made to solve these problems, and has its object to provide an image processing apparatus which enables the data transfer control in which the DMA is preformed while sharing a single memory by comprising the data transfer control apparatus which can start data transfer with high priority without waiting for the completion of the data transfer which is being executed, and more particularly, an image processing apparatus which enables the data transfer control which realizes the efficient DMA by preventing a specific DMA from being concentratively generated as well as suppressing the increase in the circuit scale.